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codes 



O 
O 

(N 



(N 



O 



> 
00 



in 
o 

O 

o 



X 
S3 



Christine A. Kelley 
Department of Mathematics 
University of Notre Dame 
Notre Dame, IN 46556, USA. 
Email: ckelleyl@nd.edu 



Abstract — Pseudocodewords of g-ary LDPC codes are exam- 
ined and the weight of a pseudocodeword on the q-ary symmetric 
channel is defined. The weight definition of a pseudocodeword 
on the AWGN channel is also extended to two-dimensional q-ary 
modulation such as q-PAM and q-PSK. The tree-based lower 
bounds on the minimum pseudocodeword weight are shown to 
also hold for q-ary LDPC codes on these channels. 

I. Introduction 

Low density parity check (LDPC) codes have been shown 
to achieve near-capacity performance over several communica- 
tion channels. Typically, they are binary linear codes described 
by sparse, randomly, generated parity-check matrices. In [3] 
and [4], the performance of non-binary LDPC codes, defined 
over larger finite fields and over integer rings, is investigated 
and compared with that of binary LDPC codes. For several 
applications such as coded-modulation, codes over higher al- 
phabets are more appropriate for system design. The popularity 
of LDPC codes is due to their efficient and simple decoding. 
Graph-based message passing iterative decoders have been 
shown to achieve near-capacity performance with complexity 
only linear in the length of the code. However, these iterative 
decoders are sub-optimal and discrepancies between iterative 
and maximum-likelihood (ML) decoding performance of short 
to moderate block length binary LDPC codes has been at- 
tributed to the presence of pseudocodewords of the LDPC 
constraint graphs (or, Tanner graphs) [8]. Analogous to the role 
of minimum Hamming distance, d m i n , in ML-decoding, the 
minimum pseudocodeword weight, w m i n , has been shown to 
be a leading predictor of performance in iterative decoding [8]. 
Furthermore, it has been observed that pseudocodewords with 
weight Wmin < (f m ; n are especially problematic for iterative 
decoding [6] . In this paper, we define pseudocodeword weights 
for q-aiy LDPC codes when the channel is a AWGN channel 
or a q-ary symmetric channel and obtain lower bounds for the 
minimum pseudocodeword weight. 

The following section shows a tree-based lower bound on 
the minimum pseudocodeword weight of binary LDPC codes. 
In Section III, the pseudocodeword weight of q-ary LDPC 
codes is defined for the AWGN and the g-ary symmetric 
channels. Subsequently, the tree-based lower bound for binary 
LDPC codes is extended to the q-ary setting. We note here 
that we restrict our analysis to pseudocodewords arising from 
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finite-degree graph covers as described in [8]. Since these 
pseudocodewords are the same as those occurring in the con- 
text of linear programming (LP) decoding, the results obtained 
here are applicable to pseudocodewords of LP decoding as 
well. Section IV summarizes the paper and outlines some 
other techniques that are being investigated for bounding the 
pseudocodeword weight of q-ary LDPC codes. 

II. Binary LDPC codes 

Definition 2.1: The tree bound of a d left (variable node) 
regular bipartite LDPC constraint graph with girth g is defined 
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Theorem 2.1: Let G be a bipartite LDPC constraint graph 
with smallest left (variable node) degree d and girth g. Then 
the minimum pseudocodeword weight w mm is lower bounded 
by 

Wmin > T(d,g). 

on the additive white Gaussian noise (AWGN) channel and the 
binary symmetric channel (BSC). 

The proof of this result is presented in [6]. The tree bound 
was originally derived by Tanner in [10] to lower-bound the 
minimum distance of the code. Since the set of pseudocode- 



words includes all codewords, we have w„ 



< d„ 



III. Non-binary LDPC codes 

Let H be a parity check matrix representing a q-ary LDPC 
code C. Thus, H is sparse in the number of non-zero entries. 
The corresponding LDPC constraint graph G that represents H 
is an incidence graph of the parity check matrix as in the binary 
case. However, each edge of G is now assigned a weight which 
is the value of the corresponding non-zero entry in H. (In [3], 
[2], LDPC codes over GF(q) are considered for transmission 
over binary modulated channels, whereas in [4], LDPC codes 
over integer rings are considered for higher-order modulation 
signal sets.) For convenience, we consider the special case 
wherein each of these edge weights are equal to one. This is 
the case when the parity check matrix has only zeros and ones. 



Furthermore, whenever the LDPC graphs have edge weights 
of unity for all the edges, we refer to such a graph as a binary 
LDPC constraint graph representing a q-ary LDPC code C. 

A. Bound on minimum distance 

We first show that if the LDPC graph corresponding to H 
is d-left (variable-node) regular, then the same tree bound of 
Theorem 12.11 holds . That is, 

Lemma 3.1: If G is a d-left regular bipartite LDPC con- 
straint graph with unity edge weights, girth g, and represents 
a q-ary LDPC code C. Then the minimum distance of the q-ary 
LDPC code C is lower bounded as 

^min 

>T(d,g). 

Proof: The proof is essentially the same as in the binary 
case. Enumerate the graph as a tree starting at an arbitrary 
variable node. Furthermore, assume that a codeword in C 
contains the root node in its support. The root variable node 
(at layer Lq of the tree) connects to d constraint nodes in the 
next layer (layer Li) of the tree. These constraint nodes are 
each connected to some sets of variable nodes in layer L2, and 
so on. Since the graph has girth g, the nodes enumerated up to 
layer L a -i when | is odd (respectively, La when ^ is even) 
are all distinct. Since the root node belongs to a codeword, 
say c, it assumes a non-zero value in c. Since the constraints 
must be satisfied at the nodes in layer L\, at least one node in 
Layer L2 for each constraint node in L\ must assume a non- 
zero value in c. (This is true under the assumption that an edge 
weight times a (non-zero) value, assigned to the corresponding 
variable node, is non-zero in the code alphabet.) 

Under the above assumption, there are at least d variable 
nodes (i.e., at least one for each node in layer L\) in layer L2 
that are non-zero in c. Continuing this argument, it is easy to 
see that the number of non-zero components in c is at least 
1 + d + d(d - 1) + . . . + d(d - l)^r when § is odd, and 
1 + d + d(d - 1) + . . . + d(d - l)** 2 +(d - l)** 4 when § 
is even. This proves the desired lower bound. ■ 

Remark 3.1: A non-zero edge-weight times a (non-zero) 
value, assigned to the corresponding variable node, may be 
zero in certain code alphabets. Since we have chosen the edge 
weights to be unity, such a case will not arise here. But also 
more generally, such cases will not arise when the alphabet and 
the arithmetic operations correspond to finite-field operations. 
However, when working over other structures, such as finite 
integer rings and more general groups, such cases could arise. 

We note here that in general this lower bound is not met 
and typically q-ary LDPC codes that have the above graph 
representation have minimum distances larger than the above 
lower bound. 

B. Pseudocodewords of q-ary LDPC codes 

Recall from [8], [6] that a pseudocodeword of an LDPC 
constraint graph G is a valid codeword in some finite cover 




input output 

Fig. 1. A (j-ary symmetric channel. 

of G. To define a pseudocodeword for a q-ary LDPC code, 
we will restrict the discussion to LDPC constraint graphs that 
have edge weights of unity among all their edges - in other 
words, binary LDPC constraint graphs that represent q-ary 
LDPC codes. A finite cover of a graph is defined in a natural 
way as in [8] wherein all edges in the finite cover also have 
an edge weight of unity. For the rest of this section, let G be 
a LDPC constraint graph of a q-ary LDPC code C of block 
length n, and let the weights on every edge of G be unity. 
We define a pseudocodeword F of G as a n x q matrix of the 
form 

/o,0 fo,l fo,2 ■ ■ ■ /o,g-l 

fl.Q fl.l fl,2 ■ ■ ■ fl.q-1 

F= . . . . . 

fn-1,0 In— 1,1 fn— 1,2 ■ ■ ■ fn-X,q—l 

where the pseudocodeword F forms a valid codeword c in a 
finite cover G of G and fa is the fraction of variable nodes 
in the i th variable node cloud, for < i < n — 1, of G that 
have the assignment (or, value) equal to j, for < j < q — 1, 
in c. 

A q-ary symmetric channel is shown in Figure ^ The input 
and the output of the channel are random variables belonging 
to a q-ary alphabet that can be denoted as {0, 1, 2, . . . , q — 1}. 
An error occurs with probability e, which is parameterized by 
the channel, and in the case of an error, it is equally probable 
for an input symbol to be altered to any one of the remaining 
symbols. 

Following the definition of pseudocodeword weight for 
the binary symmetric channel [5], we provide the following 
definition for the weight of a pseudocodeword on the q- 
ary symmetric channel. For a pseudocodeword F, let F' 
be the sub-matrix obtained by removing the first column 
in F. (Note that the first column in F contains the entries 
/o,o, /i,o, /2,o, ■ ■ ■ , /n-i,o-) Then the weight of a pseudocode- 
word F on the q-ary symmetric channel is defined as follows. 



Definition 3.1: Let e be the smallest number such that 
the sum of the e largest components in the matrix F', say, 
fiujiJi2,h,---,fie,j^ exceeds E, ¥ll ,, !r ..,,Jl ~ ko)- Then 
the weight of F on the g-ary symmetric channel is defined as 
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Note that in the above definition, none of the jVs, for 
k = 1,2, ...,e, are equal to zero, and all the i^'s, for 
k = 1, 2, ...,e, are distinct. That is, we choose at most 
one component from every row of F' when choosing the e 
largest components. The following sub-section provides an 
explanation for the above definition of weight. 

C. PSEUDOCODEWORD WEIGHT FOR q-ARY LDPC CODES 
ON THE q-ARY SYMMETRIC CHANNEL 

Suppose the all-zero codeword is sent across a q-ary sym- 
metric channel and the vector r = (ro, r%, . . . , r n -i) is 
received. Then errors occur in positions where ^ 0. Let 
S = {i\ r t ^ 0} and let S c = {i\ n = 0}. The distance 
between r and a pseudocodeword F is defined as 



ri— 1 q— 1 

d(r,F) = ^^ X (r^fc)/ 4 , fc; 

i=0 k=0 



(2) 



where x(P) is an indicator function that is equal to 1 if the 
proposition P is true and is equal to otherwise. 

The distance between r and the all-zero codeword is 

n-l 

which is the Hamming weight of r and can be obtained from 
equation (0. 

The iterative decoder chooses in favor of F instead of the 
all-zero codeword when d(r, F) < d(r, 0). That is, if 
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The condition for choosing F over the all-zero codeword 
reduces to 

{£(i-Ao) <]>>,,,} 

Hence, we define the weight of a pseudocodeword F in the 

following manner. 

Let e be the smallest number such that the sum 
of the e largest components in the matrix F , say, 

fiijiifi2,hi--> , fi*,j e > exceeds E I ^ 1 ,i 2 ,..., Ie ( 1 - ko)- Then 
the weight of F on the p-ary symmetric channel is defined as 



■ ■ + s ic 



/w.=E. 
E 



(i - Si.a) 
(i - Si.o) 



Note that in the above definition, none of the jVs, for 
k = 1,2, ...,e, are equal to zero, and all the i^'s, for 
k = 1,2, ... ,e, are distinct. That is, we choose at most one 
component in every row of F 1 when picking the e largest 
components. The received vector r = (ro,ri,... ,r n -i) that 



has the following components: = j\, r^ 2 — j2, ■ ■ ■ , Vi e = 
j e , ri = 0, for i ^ . . . , i e }, will cause the decoder to 

make an error and choose F over the all-zero codeword. 

Observe that for a codeword, the above weight definition 
reduces to the Hamming weight. If F represents a codeword 
c, then exactly w = wtn(c), the Hamming weight of c, rows 
in F 1 contain the entry 1 in some column, and the remaining 
entries in F 1 are zero. Furthermore, the matrix F has the entry 
in the first column of these w rows and has the entry 1 in 
the first column of the remaining rows. Therefore, from the 



weight definition of F, 



and the weight of F is 2e 



D. Tree Bound on the <?-ary symmetric channel 

We define the q-ary minimum pseudocodeword weight of 
G (or, minimum pseudoweight) as in the binary case, i.e., as 
the minimum weight of a pseudocodeword among all finite 
covers of G, and denote this as w m i n (G) or u> m i n when it is 
clear that we are referring to the graph G. 

Theorem 3.1: Let G be a d-left regular bipartite graph 
with girth g that represents a q-ary LDPC code C. Then 
the minimum pseudocodeword weight w m i n on the q-ary 
symmetric channel is lower bounded as 



>T(d,g) 



Proof: 




fi f 2 f r -i 

Fig. 2. Single constraint 
code. 

(1 - fifi) < 

Ej^ l ( 1 _ fj,o) 




Root 



Fig. 3. Local tree structure for a d-left regular 
graph. 

d(l-/o,o)<E ieio (l-/j.o). 
d(d-l)(l-/ ,o)<E J - 6 x 1 (l-/i,o) 



Case: | odd. Consider a single constraint node with r 
variable node neighbors as shown in Figure [2] Then, for 
i = 0, 1, . . . , r — 1 and k = 0, 1, ... ,p — 1, the following 
inequality holds: 



(1 - fi,o) < - ho) 



(3) 



Now let us consider a d-left regular LDPC constraint graph 
representing a g-ary LDPC code. We will enumerate the LDPC 
constraint graph as a tree from an arbitrary root variable node, 
as shown in Figure [5] Let F be a pseudocodeword matrix for 
this graph. Without loss of generality, let us assume that the 
component (1 — /o,o) corresponding to the root node is the 
maximum among all (1 — /i,o) over all 

Applying the inequality in at every constraint node in 
first constraint node layer of the tree, we obtain 

d(l-/o,o)< X>-/i,o), 

where Lq corresponds to variable nodes in first level of the 
tree. Subsequent application of the inequality in Q to the 
second layer of constraint nodes in the tree yields 

d(d -l)(l-/ 0j0 )< £>-/ao), 
Continuing this process until layer L a -e , we obtain 

4 

d(d- 1)^(1 -/ ,o)< E (i-Ao) 

Since the LDPC graph has girth g, the variable nodes up to 
level Lg^e are all distinct. The above inequalities yield: 

4 

[l + d + d(d-l) + ... + d(d- 1)^](1 - / ,o) 

< E (! - ho) < £ (1 " Ao) (4) 



je{o}uL u...L s _6 



all 



Let e the smallest number such that there are e maximal 
components fi 2 ,j 2 , fi 3 ,j 3 , ■ ■ -,fi e ,j e , for i 1 ,i 2 , . . . ,i e all 

distinct and 3\,3i, ■ ■ ■ ,3& £ {1, 2, . . . , 3 — 1}, in J" (the sub- 
matrix of F excluding the first column in F) such that 

/•il.Jl + f 12,32 + • • • + fi e ,j e > E ^ ~~ ^>°) 

i^{ii,i2,i3)-..,ie} 

Then, since none of the j^'s, k = 1, 2, . . . , e, are zero, we 
have 

(l-/i 1 ,0) + (l-/i 2 ,0)+...+ (l-/i«,0)>/i 1 ,i 1 +... + /i. , ie 

> E (i-Ao) 

t^{*l,t2i*3)"Mte} 

Hence we have that 

2((1 - /h.o) + (1 - A.,0) + • ■ • + (1 - fi e ,o)) 



>£(i-/i,o) 

all » 

We can then lower bound this further using the inequality 
in as 

2((1 - fn.o) + (1 - fi 2 ,o) + • • • + (1 - /< a ,o)) 



> [l + d + d(d- 1) + .. 

Since we assumed that (1 — 
(1 — fa 0) over all i, we have 



. + d(d-l)^](l-/o,o) 
/o,o) is the maximum among 



2e(l - /o,o) > 2((1 - / il)0 ) + (1 - fi 2 ,o) + • • • + (1 - /i.,o)) 

> [1 + d + d{d - 1) + . . . + d(d - 1)^](1 - / ,o) 

This yields the desired bound 

WqSC (F) = 2e>l + d + d(d-l) + ... + d(d-l) s ^. 

Since the pseudocodeword F was chosen arbitrary, we also 
have w min > 1 + d + d(d — 1) + . . . + d(d — ■ The case 
I even is treated similarly. ■ 

Since the inequality in 0, in the proof of Theorem 13. II is 
typically not tight, the above bound is rather loose. 

E. Pseudocodeword weight on the AWGN channel 

Following the definition of effective distance d 2 e ^(F : c), 
between a pseudocodeword F and a codeword c on the AWGN 
channel, presented in [5], the weight of a pseudocodeword F 
is given by d\^{F, 0). On simplifying the expression in [5], 
the weight of pseudocodeword F on the AWGN channel is 
given by 



W q -AWGN(F) 



(*) 



The above weight definition assumes g-ary pulse amplitude 
modulation, i.e., the symbols sent across the channel belong 
to the signal set {0, 1, 2, . . . , q — 1}. 

Now if we assume a two-dimensional signal set for trans- 
mission on the memoryless AWGN channel, then under the as- 
sumption that the resulting signal-space code is geometrically 
uniform [11], we can derive the weight of a pseudocodeword 
F as the effective distance of F from the all-zero codeword 
in signal space. The pseudocodeword weight of F is given by 

(R - Mf 



W q—AWGN {F) 



V 



where (x m ,y m ) is the coordinate in the two-dimensional 
signal set corresponding to the symbol m £ {0, 1, . . . , q — 1}, 



R 



EE fa™( X m + Vm) ~ x l~ Vo\> 
j m 

M = 2 Et(E fj,m x mXo) - + (E fj,mVmVo) " Vq], 



V = 4 EWE hmXm) ~ x a f + E«E hmV m ) ~ Vof] 



and j £ {0, ... n — 1}. 

Note that for q-ary pulse amplitude modulation as described 
above, this weight definition reduces to the one in (*). 

Suppose we assume q-PSK modulation, then we have x m — 
cos(22p) and y m = sin(^). Note that x = cos(0) = 1 

and yo = sin(0) = 0. In addition, R — 0. Therefore, the 
weight of a pseudocodeword F on the AWGN channel under 

q-PSK modulation is given by: w q ~AWGN{F) = where 



M ■■ 



E, ,\ > 27Tm ^ 
((2^/ 3 . m cos( )) - 1) 



Since J2™ = o T," m =o A™™ 2 > 2~2?=o(fi,i + .. - + fi,<i-l) 

Er=o c 1 - fi,o)> we have 



Wq-AWGN 



(F)> 



S"=o C 1 _ fifi) 



1 - /0,Q 

Now, the inequality from the proof of Theorem 13.11 yields 



the desired lower bound w. 



q—AWGN 



(F) > l + d + d(d-l) 



. . . + d(d — 1) 4 for the case g/2 odd. (The case g/2 even 
follows similarly.) ■ 



■2( 



E 



/i,m/j, m '(cOS( 



27r(rn - 



: ))) 



V > 27rm 

-2 > /j, m cos( ) + l 

9 



£ Tree-bound of q-ARY LDPC codes on the AWGN 

CHANNEL UNDER q-PAM 

Theorem 3.2 (q-ary pulse amplitude modulation): Let G 
be a d-left regular bipartite graph with girth q that represents 
a q-ary LDPC code C. Then the minimum pseudocodeword 
weight u> m in on the AWGN channel is lower bounded as 



>T(d,g). 



IV. Conclusions 

This paper examined the pseudocodeword weight of q- 
ary LDPC codes on the q-ary symmetric channel and the 
AWGN channel. A definition for the pseudocodeword weight 
was derived on the g-ary symmetric channel and the AWGN 
channel with two-dimensional q-ary modulation. The tree 
bound from [6] for binary LDPC codes was extended to 
the q-ary case. More sophisticated bounding techniques for 
the pseudocodeword weight of q-ary LDPC codes remains 
an open problem. It would be useful to also derive a cost- 
function of the min-sum decoder for q-ary LDPC codes to 
give an insight into which pseudocodewords are problematic 
for iterative decoding. 
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(Note that we assume a slightly unconventional definition 
of q-ary PAM in that the symbol rn is mapped to the point m 
rather than to the point 2m— 1 as in the conventional definition, 
forme {0, 1,2,..., 

Proof: Let F be a pseudocodeword in G. Without loss 
of generality, let (1 — /o.o) be the maximum of (1 — /o,i) over 
all i. We will first lower bound the weight w q _AWGN(F) as 



Wq-AWGN 



(F) = 



2~2i=0 (El=0 fi.m'nT') 2 



1-Zo.o 1 ' 

This lower bound is obtained by showing that the denomi- 
nator in the weight expression can be upper bounded by using 
the Cauchy-Schwartz inequality as follows 

n— 1 q— 1 



5^(E A™™) 2 



i=0 m=0 



<(X)(Ai + A 



n— 1 9—1 
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i=0 



i=0 m=0 



Further, since f ifl + f i<2 + . . . + f iA -i = 1 - / i>0 < 1 - /o,o, 
we obtain the lower bound in (**). 



